Aiming at the problems of simple fusion of spatio-temporal information and ignoring moving information in moving object detection based on visual saliency, a moving object detection method based on the dynamic fusion of visual saliency and moving information was proposed. Firstly, local and global saliencies of each pixel was computed by spatial characters extracted from an image, then the spatial salient map was calculated combining those saliencies by Bayesian criteria. Secondly, with the help of structured random forest, the motion boundaries were predicted to primarily orientate the moving objects, by which the motion boundary map was built. Then, according to the change of the spatial salient and motion boundary maps, the optimal fusion weights were determined dynamically. Finally, moving objects were calculated and marked by the dynamic fusion weights. The proposed approach not only inherits the advantages of saliency algorithm and moving boundary algorithm, but also overcomes their disadvantages. In the comparison experiments with the traditional background subtraction method and three-frame difference method, the detection rate and the false alarm rate of the proposed approach are improved at most more than 40%. Experimental results show that the proposed method can detect moving objects accurately and completely, and the adaptation to scene is promoted.